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Abstract. We consider Markov decision processes (MDPs) witii oj-regular spec- 
ifications given as parity objectives. We consider the problem of computing the 
set of almost-sure winning states from where the objective can be ensured with 
probability 1. The algorithms for the computation of the almost-sure winning set 
for parity objectives iteratively use the solutions for the almost-sure winning set 
for Biichi objectives (a special case of parity objectives). Our contributions are as 
follows: First, we present the first subquadratic symbolic algorithm to compute 
the almost-sure winning set for MDPs with Biichi objectives; our algorithm takes 
0{n ■ \frn) symbolic steps as compared to the previous known algorithm that 
takes O(n^) symbolic steps, where n is the number of states and ra is the num- 
ber of edges of the MDP. In practice MDPs have constant out-degree, and then our 
symbolic algorithm takes 0{n ■ ^/n) symbolic steps, as compared to the previous 
known O(n^) symbolic steps algorithm. Second, we present a new algorithm, 
namely yvin-lose algorithm, with the following two properties: (a) the algorithm 
iteratively computes subsets of the almost-sure winning set and its complement, 
as compared to all previous algorithms that discover the almost-sure winning set 
upon termination; and (b) requires 0(n ■ \/K) symbolic steps, where K is the 
maximal number of edges of strongly connected components (scc's) of the MDP. 
The win-lose algorithm requires symbolic computation of scc's. Third, we im- 
prove the algorithm for symbolic sec computation; the previous known algorithm 
takes linear symbolic steps, and our new algorithm improves the constants as- 
sociated with the linear number of steps. In the worst case the previous known 
algorithm takes 5 • n symbolic steps, whereas our new algorithm takes 4 • n sym- 
bolic steps. 



1 Introduction 

Markov decision processes. The model of systems in verification of probabilistic sys- 
tems are Markov decision processes (MDPs) that exhibit both probabilistic and non- 
deterministic behavior ifTTI . MDPs have been used to model and solve control prob- 
lems for stochastic systems (9): there, nondeterminism represents the freedom of the 
controller to choose a control action, while the probabilistic component of the behav- 
ior describes the system response to control actions. MDPs have also been adopted 
as models for concuiTent probabilistic systems [5 |, probabilistic systems operating in 
open environments 1 17|, and under-specified probabilistic systems [ 1 1. A specification 
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describes the set of desired behaviors of the system, which in the verification and con- 
trol of stochastic systems is typically an w-regular set of paths. The class of w-regular 
languages extends classical regular languages to infinite strings, and provides a robust 
specification language to express all commonly used specifications, such as safety, live- 
ness, fairness, etc ll20l . Parity objectives are a canonical way to define such w-regular 
specifications. Thus MDPs with parity objectives provide the theoretical framework to 
study problems such as the verification and control of stochastic systems. 

Qualitative and quantitative analysis. The analysis of MDPs with parity objectives 
can be classified into qualitative and quantitative analysis. Given an MDP with parity 
objective, the qualitative analysis asks for the computation of the set of states from 
where the parity objective can be ensured with probability 1 (almost-sure winning). The 
more general quantitative analysis asks for the computation of the maximal probability 
at each state with which the controller can satisfy the parity objective. 

Importance of qualitative analysis. The qualitative analysis of MDPs is an important 
problem in verification that is of interest irrespective of the quantitative analysis prob- 
lem. There are many applications where we need to know whether the correct behavior 
arises with probability 1 . For instance, when analyzing a randomized embedded sched- 
uler, we are interested in whether every thread progresses with probability 1 |7|. Even 
in settings where it suffices to satisfy certain specifications with probability p < 1, 
the correct choice of p is a challenging problem, due to the simplifications introduced 
during modeling. For example, in the analysis of randomized distributed algorithms it 
is quite common to require correctness with probability 1 (see, e.g., 1151141191 ). Fur- 
thermore, in contrast to quantitative analysis, qualitative analysis is robust to numerical 
perturbations and modeling errors in the transition probabilities, and consequently the 
algorithms for qualitative analysis are combinatorial. Finally, for MDPs with parity ob- 
jectives, the best known algorithms and all algorithms used in practice first perform 
the qualitative analysis, and then performs a quantitative analysis on the result of the 
qualitative analysis II5I6I4I . Thus qualitative analysis for MDPs with parity objectives is 
one of the most fundamental and core problems in verification of probabilistic systems. 
One of the key challenges in probabilistic verification is to obtain efficient and sym- 
bolic algorithms for qualitative analysis of MDPs with parity objectives, as symbolic 
algorithms allow to handle MDPs with a large state space. 

Previous results. The qualitative analysis for MDPs with parity objectives is achieved 
by iteratively applying solutions of the qualitative analysis of MDPs with Biichi objec- 
tives 1 5 6 4 1 . The qualitative analysis of an MDP with a parity objective with d priorities 
can be achieved by 0{d) calls to an algorithm for qualitative analysis of MDPs with 
Biichi objectives, and hence we focus on the qualitative analysis of MDPs with Biichi 
objectives. The classical algorithm for qualitative analysis for MDPs with Biichi objec- 
tives works in 0{n ■ m) time, where n is the number of states, and m is the number 
of edges of the MDP II5I6II . The classical algorithm can be implemented symbolically, 
and it takes at most O(n^) symbolic steps. An improved algorithm for the problem was 
given in |3| that works in 0{m ■ y/rn) time. The algorithm of |l3| crucially depends on 
maintaining the same number of edges in certain forward searches. Thus the algorithm 
needs to explore edges of the graph explicitly and is inherently non-symbolic. In the 
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literature, there is no symbolic subquadratic algorithm for qualitative analysis of MDPs 
with Biichi objectives. 

Our contribution. In this work our main contributions are as follows. 

1. We present a new and simpler subquadratic algorithm for qualitative analysis of 
MDPs with Biichi objectives that runs in 0(m • ^/m) time, and show that the al- 
gorithm can be implemented symbohcally. The symbolic algorithm takes at most 
0{n ■ \pm) symbolic steps, and thus we obtain the first symbolic subquadratic al- 
gorithm. In practice, MDPs often have constant out-degree: for example, see [S) 
for MDPs with large state space but constant number of actions, or 091161 for ex- 
amples from inventory management where MDPs have constant number of actions 
(the number of actions correspond to the out-degree of MDPs). For MDPs with 
constant out-degree our new symbolic algorithm takes 0(n ■ y/n) symbolic steps, 
as compared to 0{n^) symbolic steps of the previous best known algorithm. 

2. All previous algorithms for the qualitative analysis of MDPs with Biichi objectives 
iteratively discover states that are guaranteed to be not almost-sure winning, and 
only when the algorithm terminates the almost-sure winning set is discovered. We 
present a new algorithm (namely win-lose algorithm) that iteratively discovers both 
states in the almost-sure winning set and its complement. Thus if the problem is to 
decide whether a given state s is almost-sure winning, and the state s is almost-sure 
winning, then the win-lose algorithm can stop at an intermediate iteration unlike all 
the previous algorithms. Our algorithm works in time 0{\/ KE-m) time, where Ke 
is the maximal number of edges of any sec of the MDP (in this paper we write sec 
for maximal sec). We also show that the win-lose algorithm can be implemented 
symbolically, and it takes at most 0{\/Ke ■ n) symbolic steps. 

3. Our win-lose algorithm requires to compute the sec decomposition of a graph in 
0{n) symbolic steps. The sec decomposition problem is one of the most fundamen- 
tal problem in the algorithmic study of graph problems. The symbolic sec decom- 
position problem has many other applications in verification: for example, check- 
ing emptiness of w-automata, and bad-cycle detection problems in model checking, 
see lIJl for other applications. An 0{n ■ log n) symbolic step algorithm for sec de- 
composition was presented in f2 |, and the algorithm was improved in |10|. The 
algorithm of |10| is a linear symbolic step sec decomposition algorithm that re- 
quires at most min{ ^ ■ n^b ■ D ■ N + N } symbolic steps, where D is the diameter 
of the graph, and N is the number of see's of the graph. We present an improved 
version of the symbolic sec decomposition algorithm. Our algorithm improves the 
constants of the number of the linear symbolic steps. Our algorithm requires at most 
min{ 3 • n + A^, 5 • D* + A^ } symbolic steps, where D* is the sum of the diameters 
of the see's of the graph. Thus, in the worst case, the algorithm of 1 10 1 requires 5 • n 
symbolic steps, whereas our algorithm requires 4 • n symbolic steps. Moreover, the 
number of symbolic steps of our algorithm is always bounded by the number of 
symbolic steps of the algorithm of 1 10| (i.e. our algorithm is never worse). 

Our experimental results show that our new algorithms perform better than the previous 
known algorithms both for qualitative analysis of MDPs with Biichi objectives and 
symbolic sec computation. 
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2 Definitions 



Markov decision processes (MDPs). A Markov decision process (MDP) G = 
{{S, E), {Si, Sp),5) consists of a directed graph {S, E), a partition {Si,Sp) of the_^- 
nite set S of states, and a probabilistic transition function &: Sp — > 'D{S), where V^S) 
denotes the set of probability distributions over the state space S. The states in 5*1 are 
the player-1 states, where player 1 decides the successor state, and the states in Sp are 
the probabilistic (or random) states, where the successor state is chosen according to 
the probabilistic transition function 5. We assume that for s e Sp and t e S", we have 
(s, t) & E iff 6{s){t) > 0, and we often write S{s, t) for 5{s){t). For a state s e S, we 
write E{s) to denote the set { t e 5 | (s, t) € -E } of possible successors. For technical 
convenience we assume that every state in the graph {S, E) has at least one outgoing 
edge, i.e., E{s) ^ for all s e S". 

Plays and strategies. An infinite path, or a play, of the game graph G is an infinite 
sequence lo — {sq, si, S2, ■ ■ ■) of states such that {s^, s^+i) € E for all fc e N. We 
write f2 for the set of all plays, and for a state s G S", we write Sis ^ for the set of 
plays that start from the state s. A strategy for player 1 is a function a: S* ■ Si T^{S) 
that chooses the probability distribution over the successor states for all finite sequences 
w € S* ■ Si of states ending in a player-1 state (the sequence represents a prefix of 
a play). A strategy must respect the edge relation: for all w ^ S* and s G Si, if 
a{w ■ s){t) > 0, then t e E{s). A strategy is deterministic (pure) if it chooses a 
unique successor for all histories (rather than a probability distribution), otherwise it 
is randomized. Player 1 follows the strategy u if in each player-1 move, given that the 
current history of the game is to G S* ■ Si, she chooses the next state according to 
a{w). We denote by S the set of all strategies for player 1. A memoryless player-1 
strategy does not depend on the history of the play but only on the current state; i.e., for 
all w, w' G S* and for all s G S*! we have (j{w ■ s) = (t{w' ■ s). A memoryless strategy 
can be represented as a function a: Si — ^ T^iS), and a pure memoryless strategy can 
be represented as a : Si S. 

Once a starting state s G 5 and a strategy a E S is fixed, the outcome of the MDP 
is a random walk for which the probabilities of events are uniquely defined, where 
an event AC n is a measurable set of plays. For a state s £ S and an event A C f], 
we write Pr^ (A) for the probability that a play belongs to A if the game starts from the 
state s and player 1 follows the strategy a. 

Objectives. We specify objectives for the player 1 by providing a set of winning plays 
<P C SI. We say that a play oj satisfies the objective if cj G "P. We consider oj- 
regular objectives |20|, specified as parity conditions. We also consider the special case 
of Biichi objectives. 

- Biichi objectives. Let T be a set of target states. For a play uj ~ {sq, si, . . .) G S7, 
we define Inf (cj) = { s G 5* | Sfc = s for infinitely many fc } to be the set of states 
that occur infinitely often in oj. The Biichi objectives require that some state of T 
be visited infinitely often, and defines the set of winning plays Buchi(T) = { w G 
12 I Inf (w) n T 7^ }. 

- Parity objectives. For c,d G N, we write [c.d] — { c,c + 1, . . . ,d }. Let p: 
S — > [0..d] be a function that assigns a priority p{s) to every state s G S', 
where d G N. The parity objective is defined as Parity(p) — { uj £ S7 \ 
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min (p(Inf (w))) is even }. In other words, the parity objective requires that the 
minimum priority visited infinitely often is even. In the sequel we will use ^ to 
denote parity objectives. 

Qualitative analysis: almost-sure winning. Given a player- 1 objective a strategy 
(7 G S is almost-sure winning for player 1 from the state s if Pr^ (^) — 1. The almost- 
sure winning set ((!)) almost (^) for player 1 is the set of states from which player 1 has 
an almost-sure winning strategy. The qualitative analysis of MDPs correspond to the 
computation of the almost-sure winning set for a given objective <P. It follows from the 
results of 1 5 6 1 that for all MDPs and all reachability and parity objectives, if there is an 
almost-sure winning strategy, then there is a memoryless almost-sure winning strategy. 
The qualitative analysis of MDPs with parity objectives is achieved by iteratively ap- 
plying the solutions of qualitative analysis for MDPs with Biichi objectives |6j4J, and 
hence in this work we will focus on qualitative analysis for Biichi objectives. 

Theorem 1 ( II5I6I ). For all MDPs G, and all reachability and parity objectives <P, there 
exists a pure memoryless strategy cr* such that for all s G {{^)) aimosti^) we have 
Pr:*(^>) = l. 

Sec and bottom sec. Given a graph G = (S*, £'), a set C of states is an sec if for all 
s,t G C there is a path from s to t going through states in G. An sec C is a bottom sec 
if for all s e C all out-going edges are in C, i.e., E{s) C G. 

Markov chains, closed recurrent sets. A Markov chain is a special case of MDP 
with Si = 0, and hence for simplicity a Markov chain is a tuple {{S,E),6) with a 
probabiHstic transition function 6 : S ^ ^(•5), and {s,t) € E iff S{s,t) > 0. A 
closed recurrent set C of a Markov chain is a bottom sec in the graph {S,E). Let 
C = Uc is closed recuiTent ^- foUows from the rcsults on Markov chains [ 1 3 1 that for all 
s E S, the set C is reached with probability 1 in finite time, and for all G such that G is 
closed recurrent, for all s e C and for all t E G,if the starting state is s, then the state 
t is visited infinitely often with probability 1 . 

Markov chain from a MDP and memoryless strategy. Given a MDP G — 
{{S, E), 5p), 6) and a memoryless strategy ct» : 5i — > ^{S) we obtain a Markov 
chain G' = {{S,E'),5') as follows: E' = En{Sp x S)U {{s,t) \ s € Si,a^{s)it) > 
}; and (5'(s, t) = 5{s, t) for s £ Sp, and S'{s, t) = a{s)(t) for s G Si and t € eIs). 
We will denote by G^, the Markov chain obtained from an MDP G by fixing a memo- 
ryless strategy ci* in the MDP. 

Symbolic encoding of an MDP. All algorithms of the paper will only depend on the 
graph {S, E) of the MDP and the partition (5'i, Sp), and not on the probabilistic tran- 
sition function 5. Thus the symbolic encoding of an MDP is obtained as the standard 
encoding of a transition system (with an Obdd |18 |), with one additional bit, and the 
bit denotes whether a state belongs to 5*1 or Sp. 

3 Symbolic Algorithms for Biichi Objectives 

In this section we will present a new improved algorithm for the qualitative analysis 
of MDPs with Biichi objectives, and then present a symbolic implementation of the 
algorithm. Thus we obtain the first symbolic subquadratic algorithm for the problem. 
We start with the notion of attractors that is crucial for our algorithm. 
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Random and player 1 attractor. Given an MDP G, let [/ C be a subset of states. The 
random attractor AttrB,{U) is defined inductively as follows: Xq = U, and for i > 0, 

let ^ X^U{s e Sp \ E{s) nX,^(!)}U{seSi\E{s)CX^}.ln other words, 
Xi+i consists of (a) states in Xi, (b) player- 1 states whose all successors are in Xi and 
(c) random states that have at least one edge to Xi. Then Attrii{U) = [Ji^QXi. The 
definition of player-1 attractor Attri{U) is analogous and is obtained by exchanging 
the role of random states and player 1 states in the above definition. 

Property of attractors. Given an MDP G, and set U of states, let A = Attru{U). 
Then from A player 1 cannot force to avoid U, in other words, for all states in A and for 
all player 1 strategies, the set U is reached with positive probability. For A = Attri (U) 
there is a player 1 memoryless strategy to ensure that the set U is reached with certainty. 
The computation of random and player 1 attractor is the computation of alternating 
reachability and can be achieved in 0(m) time lil2J . and can be achieved in 0{n) 
symbolic steps. 

3.1 A new subquadratic algorithm 

The classical algorithm for computing the almost-sure winning set in MDPs with Biichi 
objectives has 0{n ■ m) running time, and the symbolic implementation of the algo- 
rithm takes at most 0{ii?) symbolic steps. A subquadratic algorithm, with Oijn ■ -y/m) 
running time, for the problem was presented in [3 1. The algorithm of [3] uses a mix of 
backward exploration and forward exploration. Every forward exploration step consists 
of executing a set of DFSs (depth first searches) simultaneously for a specified number 
of edges, and must maintain the exploration of the same number of edges in each of 
the DFSs. The algorithm thus depends crucially on maintaining the number of edges 
traversed explicitly, and hence the algorithm has no symbolic implementation. In this 
section we present a new subquadratic algorithm to compute ((l))a;,„ost(Buchi(T)). 
The algorithm is simpler as compared to the algorithm of [3 1 and we will show that our 
new algorithm can be implemented symbolically. Our new algorithm has some similar 
ideas as the algorithm of 1 3 1 in mixing backward and forward exploration, but the key 
difference is that the new algorithm never stops the forward exploration after a certain 
number of edges, and hence need not maintain the traversed edges explicitly. Thus the 
new algorithm is simpler, and our correctness and running time analysis proofs are dif- 
ferent. We show that our new algorithm works in 0(to • y/rn) time, and requires at most 
0{n ■ y/rn) symbolic steps. 

Improved algorithm for almost-sure Biichi. Our algorithm iteratively removes states 
from the graph, until the almost-sure winning set is computed. At iteration i, we denote 
the remaining subgraph as (5^, Ei), where 5*^ is the set of remaining states, Ei is the 
set of remaining edges, and the set of remaining target states as Ti (i.e., Ti = SiC\ T). 
The set of states removed will be denoted by Z^, i.e.. Si = S \ Zi. The algorithm will 
ensure that (a) Zi C S \ {{!)) almost (^^chi{T)); and (b) for all s e Si Ci Sp we have 
E{s) n Zi = 0. In every iteration the algorithm identifies a set Qi of states such that 
there is no path from Qi to the set Ti. Hence clearly Qi C S\ ((l))a(most (Buchi(T)). By 
the random attractor property from Attrp{Qi) the set Qi is reached with positive prob- 
ability against any strategy for player 1. The algorithm maintains the set L^+i of states 
that were removed from the graph since (and including) the last iteration of Case 1, 
and the set Ji+i of states that lost an edge to states removed from the graph since the 
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last iteration of Case 1. Initially Lq := -^o := 0, Zq := 0, and let i := and we de- 
scribe the iteration i of our algorithm, and we call our algorithm ImprAlgo (Improved 
Algorithm) and the formal pseudocode is in the appendix. 

1. Case 1. If ((I J, I > V^) ori = 0), then 

(a) Let Yi be the set of states that can reach the current target set Tj (this can be 
computed in 0{m) time by a graph reachability algorithm). 

(b) Let Qi := Si \ Yi, i.e., there is no path from Qi to Ti. 

(c) Zj+i := Zi U AttrniQi)- The set AttrniQi) is removed from the graph. 

(d) The set ij+i is the set of states removed from the graph in this iteration (i.e., 
-Li+i := Attrii{Qi)) and J,+i be the set of states in the remaining graph with 
an edge to Li+i. 

(e) If Qi is empty, the algorithm stops, otherwise i := i + 1 and go to the next 
iteration. 

2. Case 2. Else {\Ji\ < y/m), then 

(a) We do a lock-step search from every state ,s in J, as follows: we do a DPS from 
s and (a) if the DPS tree reaches a state in Ti, then we stop the DPS search from 
s; and (b) if the DPS is completed without reaching a state in Ti, then we stop 
the entire lock-step search, and all states in the DPS tree are identified as Qi. 
The set Attrjf (Qi) is removed from the graph and Zi^i := Zi U Attrj^iQi). If 
DPS searches from all states s in Ji reach the set Ti, then the algorithm stops. 

(b) The set L^+i is the set of states removed from the graph since the last iter- 
ation of Case 1 (i.e., Xj+i := Li U AttrR{Qi), where Qi is the DPS tree 
that stopped without reaching Tj in the previous step of this iteration) and 
Jj_i_i be the set of states in the remaining graph with an edge to i^+i, i.e., 
Jj+i := {Ji \ Attrji{Qi)) U Xj, where Xi is the subset of states of Si with an 
edge to AttrR{Qi). 

(c) i := i + 1 and go to the next iteration. 

Correctness and running time analysis. We first prove the correctness of the algo- 
rithm. 

Lemma 1. Algorithm ImprAlgo correctly computes the set {{!)) almost {BUchi{T)). 

Proof. We consider an iteration i of the algorithm. Recall that in this iteration Yi is 
the set of states that can reach Ti and Qi is the set of states with no path to Ti. Thus 
the algorithm ensures that in every iteration i, for the set of states Qi identified by 
the algorithm there is no path to the set Tj, and hence from Qi the set Tj cannot be 
reached with positive probability. Clearly, from Qi the set Ti cannot be reached with 
probabihty 1. Since from AttrniQi) the set Qi is reached with positive probability 
against all strategies for player 1, it follows that from Attrn{Qi) the set Tj cannot be 
ensured to be reached with probability 1 . Thus for the set Zi of removed states we have 
S \ ((1)) almost (Buchi(T)). It follows that all the states removed by the algorithm 
over all iterations are not part of the almost-sure winning set. 

To complete the correctness argument we show that when the algorithm stops, the 
remaining set is ((l))aimost(Buchi(T)). When the algorithm stops, let S'* be the set of 
remaining states and T* be the set of remaining target states. It follows from above 
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that 5 \ 5, C S \ ((l))aimost(Buchi(r)) and to complete the proof we show 5* C 
((l))aimosf (Buchi(T)). The following assertions hold: (a) for all s € S*, n S'p we have 
E{s) C St,, and (b) for all states s € S'* there is a path to the set T*. We prove (a) as 
follows: whenever the algorithm removes a set Zi, it is a random attractor, and thus if a 
state s e 5* n S'p has an edge (s, t) with t £ S \ S*,, then s would have been included 
in S \ S^, and thus (a) follows. We prove (b) as follows: (i) If the algorithm stops in 
Case 1, then Qi — 0, and it follows that every state in 5* can reach T^,. (ii) We now 
consider the case when the algorithm stops in Case 2: In this case every state in Ji has 
a path to Tj = T*, this is because if there is a state s in Ji with no path to Ti, then 
the DFS tree from s would have been identified as Qi in step 2 (a) and the algorithm 
would not have stopped. It follows that there is no bottom sec in the graph induced 
by 5* that does not intersect with T*: because if there is a bottom sec that does not 
contain a state from Jj and also does not contain a target state, then it would have been 
identified in the last iteration of Case 1 . Since every state in S** has an out-going edge, 
it follows every state in 5* has a path to T,. Hence (b) follows. Consider a shortest 
path (or the BFS tree) from all states in 5* to T*, and for a state s e H Si, let s' 
be the successor for the shortest path, and we consider the pure memoryless strategy 
(7* that chooses the shortest path successor for all states s G (5* \ T*) n ^i, and in 
states in T^, n 5*1 choose any successor in 5*. Let £ = 15**1 and let a be the minimum 
of the positive transition probability of the MDP. For all states s e 5*, the probability 
that T* is reached within £ steps is at least a^, and it follows that the probability that 
T* is not reached within k x £ steps is at most (1 — a^)*^, and this goes to as 
goes to 00. It follows that for all s G 5* the pure memoryless strategy cr* ensures 
that T* is reached with probability 1. Moreover, the strategy ensures that S* is never 
left, and hence it follows that T* is visited infinitely often with probability 1. It follows 
thatS*, C ((l))<ji„ost(Buchi(T*)) C ((l))a;„ost(Biichi(T)) and hence the correctness 
follows. I 

We now analyze the rurming time of the algorithm. 

Lemma 2. Given an MDP G with m edges, Algorithm ImprAlgo takes 0{m ■ ^Jm) 
time. 

Proof. The total work of the algorithm, when Case 1 is executed, over all iterations is 
at most 0{\Jm ■ m): this follows because between two iterations of Case 1 at least ^/m 
edges must have been removed from the graph (since \Ji\ > y/m every time Case 1 is 
executed other than the case when i = 0), and hence Case 1 can be executed at most 
misfm = \fm times. Since each iteration can be achieved in 0(rn) time, the 0(m • 
yjm) bound for Case 1 follows. We now show that the total work of the algorithm, when 
Case 2 is executed, over all iterations is at most 0{yjrn-m) . The argument is as follows: 
consider an iteration i such that Case 2 is executed. Then we have | Ji| < \fm. Let Qi 
be the DFS tree in iteration i while executing Case 2, and let E{Qi) = \JseQ^E{s). 
The lock-step search ensures that the number of edges explored in this iteration is at 
most \ Ji\ ■ \E{Qi)\ < sjrh x \E{Qi)\. Since Qi is removed from the graph we charge 
the work of ^/m ■ \E[Q.i) \ to edges in E{Qi), charging work y/m to each edge. Since 
there are at most m edges, the total charge of the work over all iterations when Case 2 
is executed is at most 0{m ■ y/m). Note that if instead of ^/m we would have used a 
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bound k in distinguishing Case 1 and Case 2, we would have achieved a running time 
bound of 0(m^/fc + m-A;), which is optimized by fc = \/m- Our desired result follows. 
I 

This gives us the following result. 

Theorem 2. Given an MDP G and a set T of target states, the algorithm ImprAlgo 
correctly computes the set {{1}) aimost{BUchi{T)) in time 0{m ■ y/m). 

3.2 Symbolic implementation of ImprAlgo 

In this subsection we will a present symbolic implementation of each of the steps of 
algorithm ImprAlgo. The symbolic algorithm depends on the following symbolic op- 
erations that can be easily achieved with an Obdd implementation. For a set X C 5 of 
states, let 

Pre(X) = { s e S I E{s) n X ^ }; Post(X) = {t€S\t€ U^eJC E{s) }; 
CPre(X) = { s e I E{s) nX^0}U{se5i| E{s) C X }. 

In other words, Pre(X) is the predecessors of states in X; Post(X) is the successors of 
states in X; and CPre(X) is the set of states Y such that for every random state in Y 
there is a successor in X, and for every player 1 state in Y all successors are in Y. 

We now present a symboUc version of ImprAlgo. For the symbolic version the 
basic steps are as follows: (i) Case 1 of the algorithm is same as Case 1 of ImprAlgo, 
and (ii) Case 2 is similar to Case 2 of ImprAlgo, and the only change in Case 2 
is instead of lock-step search exploring the same number of edges, we have lock-step 
search that executes the same number of symbolic steps. The details of the symbolic 
implementation are as follows, and we will refer to the algorithm as SymbImprAlgo. 

1. Case 1. In Case 1(a) we need to compute reachability to a target set T. The symbolic 
implementation is standard and done as follows: Xq = T and Xj+i := Xj U 
Pre(Xj) until X^+i = Xi. The computation of the random attractor is also standard 
and is achieved as above replacing Pre by CP re. It follows that every iteration of 
Case 1 can be achieved in 0{n) symboUc steps. 

2. Case 2. For analysis of Case 2 we present a symbolic implementation of the lock- 
step forward search. The lock-step ensures that each search executes the same num- 
ber of symboUc steps. The implementation of the forward search from a state s in 
iteration i is achieved as follows: Po { s } ™d Pj+i := Pj U Post(Pj) unless 
Pj+i = Pj or Pj nTi^$. If Pj n Ti ^ 0, then the forward search is stopped from 
s. If Pj+i = Pj and Pj n Ti = 0, then we have identified that there is no path from 
states in Pj to T^. 

3. Symbolic computation of cardinality of sets. The other key operation required by 
the algorithm is determining whether the size of set Ji is at least y/m or not. Below 
we describe the details of this symbolic operation. 

Symbolic computation of cardinality. Given a symbolic description of a set X and 
a number k, our goal is to determine whether |X| < k. A naive way is to check for 
each state, whether it belongs to X. But this takes time proportional to the size of state 
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space and also is not symbolic. We require a procedure that uses the structure of a BDD 
and directly finds the states which this BDD represents. It should also take into account 
that if more than k states are already found, then no more computation is required. 
We present the following procedure to accomplish the same. A cube of a BDD is a 
path from root node to leaf node where the leaf node is the constant 1 (i.e. true). Thus, 
each cube represents a set of states present in the BDD which are exactly the states 
found by doing every possible assignment of the variables not occurring in the cube. For 
an explicit implementation: consider a procedure that uses Cudd_ForEachCube (from 
CUDD package, see fTF] for symbolic implementation) to iterate over the cubes of a 
given Obdd in the same manner the successor function works on a binary tree. If I is 
the number of variables not occurring in a particular cube, we get 2' states from that 
cube which are part of the Obdd. We keep on summing up all such states until they 
exceed k. If it does exceed, we stop and say that \X\ > k. Else we terminate when 
we have exhausted all cubes and we get \X\ < k. Thus we require min(fc, \BDD{X)\) 
symbolic steps, where BDD{X) is the size of the Obdd of X. We also note, that 
this method operates on Obdds that represent set of states, and these Obdds only 
use log(n) variables compared to 2 • log(n) variables used by Obdds representing 
transitions (edge relation). Hence, the operations mentioned are cheaper as compared 
to Pre and Post computations. 

Correctness and runtime analysis. The correctness of SymbImprAlgo is estab- 
lished following the correctness arguments for algorithm ImprAlgo. We now analyze 
the worst case number of symbolic steps. The total number of symbolic steps executed 
by Case 1 over all iterations is 0{n ■ y/m) since between two executions of Case 1 at 
least y/m edges are removed, and every execution is achieved in 0{n) symbolic steps. 
The work done for the symbolic cardinality computation is charged to the edges already 
removed from the graph, and hence the total number of symbolic steps over all itera- 
tions for the size computations is 0{m). We now show that the total number of symbolic 
steps executed over all iterations of Case 2 is 0{n ■ y/m). The analysis is achieved as 
follows. Consider an iteration i of Case 2, and let the number of states removed in the 
iteration be m. Then the number of symbolic steps executed in this iteration for each 
of the forward search is at most Ui, and since | Ji| < \/m, it follows that the number of 
symbolic steps executed is at most • y/rri. Since we remove states, we charge each 
state removed from the graph with y/rri symbolic steps for the total ri; • y/rri symbolic 
steps. Since there are at most n states, the total charge of symbolic steps over all itera- 
tions is 0(n • y/m). Thus it follows that we have a symbolic algorithm to compute the 
almost-sure winning set for MDPs with Biichi objectives in 0(n • y/m) symbolic steps. 

Theorem 3. Given an MDP G and a set T of target states, the symbolic algorithm 
SymbImprAlgo correctly computes {{!)) almost iBuchi{T)) in 0(n ■ \/rn) symbolic 
steps. 

Remark 1. In many practical cases, MDPs have constant out-degree and hence we ob- 
tain a symbolic algorithm that works m 0{n ■ y/n) symbolic steps, as compared to the 
previous known (symbolic implementation of the classical) algorithm that takes 0{n^) 
symbolic steps. 
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3.3 Optimized SymbImprAlgo 



In the worst case, the SymbImprAlgo algorithm takes 0{n ■ \/rn) steps. However it 
is easy to construct a family of MDPs with n states and 0{n) edges, where the classical 
algorithm takes 0{n) symbolic steps, whereas SymbImprAlgo takes 0{n- ^/n) sym- 
bolic steps. One approach to obtain an algorithm that takes at most 0{n ■ ^Jn) symbolic 
steps and no more than linearly many symbolic steps of the classical algorithm is to 
dovetail (or run in lock-step) the classical algorithm and SymbImprAlgo, and stop 
when either of them stops. This approach will take time at least twice the minimum 
running time of the classical algorithm and SymbImprAlgo. We show that a much 
smarter dovetailing is possible (at the level of each iteration). We now present the smart 
dovetailing algorithm, and we call the algorithm SmDvSymbImprAlgo. The basic 
change is in Case 2 of SymbImprAlgo. We now describe the changes in Case 2: 

- At the beginning of an execution of Case 2 at iteration i such that the last execution 

was Case 1, we initialize a set Ui to T^. Every time a post computation (Post(Pj)) is 
done, we update t/j by t/j+i := J/j U Pre(J7i) (this is the backward exploration step 
of the classical algorithm and it is dovetailed with the forward exploration step in 
every iteration). For the forward exploration step, we continue the computation of 
Pj unless Pj+i = Pj or Pj n C/j ^ (i.e., SymbImprAlgo checked the emptiness 
of intersection with Tj, whereas in SmDvSymbImprAlgo the emptiness of the 
intersection is checked with Ui). If C/i+i = Ui (i.e., a fixpoint is reached), then 
Si \ Ui and its random attractor is removed from the graph. 

Correctness and symbolic steps analysis. Details are given in appendix and we have 
the following result. 

Theorem 4. Given an MDP G and a set T of target states, the symbolic algorithm 
SmDvSymbImprAlgo correctly computes {{!)) aimost{Buchi{T)) and requires at 
most 

min{ 2 • SymbStep(SYMBlMPRALGO), 2 • SymbStep(CLASSiCAL) -|- 0(m) } 

symbolic steps, where SymbStep is the number of symbolic steps of an algorithm. 

Observe that it is possible that the number of symbolic steps and running time of 
SmDvSymbImprAlgo is smaller than both SymbImprAlgo and Classical (in 
contrast to a simple dovetailing of SymbImprAlgo and Classical, where the run- 
ning time and symbolic steps is twice that of the minimum). It is straightforward to con- 
struct a family of examples where SmDvSymbImprAlgo takes linear (0(n)) sym- 
bolic steps, however both CLASSICAL and SymbImprAlgo take at least 0{n ■ s/n) 
symbolic steps. 

4 The Win-Lose Algorithm 

AU the algorithms known for computing the almost-sure winning set (including the al- 
gorithms presented in the previous section) iteratively compute the set of states from 
where it is guaranteed that there is no almost-sure winning strategy for the player. The 
almost-sure wirming set is discovered only when the algorithm stops. In this section, 
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first we will present an algorithm that iteratively computes two sets Wi and W2, where 
Wi is a subset of the almost-sure winning set, and W2 is a subset of the complement of 
the almost-sure winning set. The algorithm has 0(K ■ m) running time, where K is the 
size of the maximal strongly connected component (sec) of the graph of the MDP. We 
then present an improved version of the algorithm, using the techniques to obtain iM- 
prAlgo from the classical algorithm, and finally present the symbolic implementation 
of the new algorithm. 

4.1 The basic win-lose algorithm 

The basic steps of the new algorithm are as follows. The algorithm maintains Wi and 
W2, that are guaranteed to be subsets of the almost-sure winning set and its complement 
respectively. Initially Wi = and W2 = 0. We also maintain that Wi = Attri{Wi) 
and W2 = AttrR{W2). We denote by W the union of Wi and W2. We describe an 
iteration of the algorithm and we will refer to the algorithm as the WinLose algorithm 
(formal pseudocode in the appendix). 

1. Step 1. Compute the sec decomposition of the remaining graph of the MDP, i.e., 
sec decomposition of the MDP graph induced by S* \ W. 

2. Step 2. For every bottom sec C in the remaining graph: if C n Pre(Vl^i) 7^ or 
C n T ^ 0, then M^i = Attn {Wi U C) ; else W2 = AttrR {W2 U C), and the states 
in Wi and W2 are removed from the graph. 

The stopping criterion is as follows; the algorithm stops when W = S. Observe that in 
each iteration, a set C of states is included in either Wi or W2, and hence W grows in 
each iteration. 

Correctness of the algorithm. Note that in Step 2 we ensure that Attri{Wi) = W\ 
and AttrR{W2) — W2, and hence in the remaining graph there is no state of player 1 
with an edge to Wi and no random state with an edge to 14^2 • We show by induction that 
after every iteration T^i C ((1)) (Buchi(T)) and C S\{{l))aimoA^nchi{T)). 
The base case (with Wi = W2 = 0) follows trivially. We prove the inductive case 
considering the following two cases. 

1. Consider a bottom sec C in the remaining graph such that C n Pre(Wi) 7^ or 
CnT ^ 0. Consider the randomized memoryless strategy a for the player that plays 
all edges in C uniformly at random, i.e., for s e C we have <T{s){t) = i^^^^"!^^^ for 
t e E{s)nC. If CnPre(T4^i) ^ 0, then the strategy ensures that Wi is reached with 
probability 1, since Wi C {{!)) almost {^^chi{T)) by inductive hypothesis it follows 
C C {{!)) aimost{BixchiiT)). Hence AttniWiUC) C ((1)) (Buchi(T)). If 
C n T 7^ 0, then since there is no edge from random states to W2, it follows that 
under the randomized memoryless strategy a, the set C is a closed recurrent set 
of the resulting Markov chain, and hence every state is visited infinitely often with 
probability 1. Since C n T 7^ 0, it follows that C C ((1)) (Buchi(T)), and 
hence Attn{Wi UC)C ((1)) (Buchi(T)). 

2. Consider a bottom sec C in the remaining graph such that C fl Pre{Wi) = and 
C n T = 0. Then consider any strategy for player 1: (a) If a play starting from a 
state in C stays in the remaining graph, then since C is a bottom sec, it follows 
that the play stays in C with probability 1. Since C fl T = it follows that T is 
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never visited, (b) If a play leaves C (note that C is a bottom sec of the remaining 
graph and not the original graph, and hence a play may leave C), then since C fl 
Pre(T4^i) ~ 0, it follows that the play reaches W2, and by hypothesis W2 C 5' \ 
((l))aimost(Buchi(r)). In either case it follows that C C S'\((l))ajmo.f (Biichi(T)). 
It follows that AttrR{W2 U C) C 5 \ ((l))ai™o«t(Buchi(T)). 

The correctness of the algorithm follows as when the algorithm stops we have W\ U 

W2 = S. 

Running time analysis. In each iteration of the algorithm at least one state is removed 
from the graph, and every iteration takes at most 0(m) time: in every iteration, the 
sec decomposition of step 1 and the attractor computation in step 2 can be achieved in 
0{m) time. Hence the naive running of the algorithm is 0{n-m). The desired 0{K-m) 
bound is achieved by considering the standard technique of miming the algorithm on 
the sec decomposition of the MDP. In other words, we first compute the sec of the 
graph of the MDP, and then proceed bottom up computing the partition Wi and W2 for 
an sec C once the partition is computed for all states below the sec. Observe that the 
above correctness arguments are still valid. The running time analysis is as follows: let 
^ be the number of scc's of the graph, and let and be the number of states and 
edges of the i-th sec. Let K = max{ rij | 1 < i < f }. Our algorithm runs in time 
0{m) + E-=i 0{ni ■ rrii) < 0{m) + ^ti 0{K ■ rm) = 0{K ■ m). 

Theorem 5. Given an MDP with a Biichi objective, the WinLose algorithm iteratively 
computes the subsets of the almost-sure winning set and its complement, and in the 
end correctly computes the set {{!)) aimostiBuchi{T)) and the algorithm runs in time 
0{Ks ■ m), where Kg is the maximum number of states in an sec of the graph of the 
MDP. 

4.2 Improved WinLose algorithm and symbolic implementation 

Improved WinLose algorithm. The improved version of the WinLose algorithm 
performs a forward exploration to obtain a bottom sec like Case 2 of ImprAlgo. At 
iteration i, we denote the remaining subgraph as {Si, Ei), where Si is the set of re- 
maining states, and Ei is the set of remaining edges. The set of states removed will be 
denoted by Zi, i.e.. Si = S \ Zi, and Zi is the union of Wi and W2. In every iteration 
the algorithm identifies a set Cj of states such that Cj is a bottom sec in the remain- 
ing graph, and then it follows the steps of the WinLose algorithm. We will consider 
two cases. The algorithm maintains the set Lj+i of states that were removed from the 
graph since (and including) the last iteration of Case 1, and the set Jj+i of states that 
lost an edge to states removed from the graph since the last iteration of Case 1. Initially 
Jo := Lq := Zq := Wi := W2 := 0, and let i := and we describe the iteration 
i of our algorithm. We call our algorithm ImprWinLose (formal pseudocode in the 
appendix). 

1. Case 1. If ((I Jil > ^/m) or i = 0), then 

(a) Compute the sec decomposition of the remaining graph. 

(b) For each bottom sec if Ci n T ^ or Cj n Pre(VFi) ^ 0, then Wi := 
AttniWi U d), else W2 := AttrR{W2 U d). 
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(c) Zi^i := W\ U W2. The set Zi+i \ Zi is removed from the graph. 

(d) The set L^+i is the set of states removed from the graph in this iteration and 
Ji+i be the set of states in the remaining graph with an edge to L^+i. 

(e) If Zi is S, the algorithm stops, otherwise i := i + 1 and go to the next iteration. 
2. Case 2. Else (| J^j < -/rn), then 

(a) Consider the set Ji to be the set of vertices in the graph that lost an edge to the 
states removed since the last iteration that executed Case 1 . 

(b) We do a lock-step search from every state s in Ji as follows: we do a DPS from 
s, until the DPS stops. Once the DPS stops we have identified a bottom sec Ci. 

(c) If C, n T =^ or a n Pre{Wi) ^ 0, then Wi := Attri{Wi U C,), else 
W2 := AttrR{W2 U a). 

(d) Zi+i := Wi U W2. The set Z^+i \ Zi is removed from the graph. 

(e) The set L^+i is the set of states removed from the graph since the last iteration 
of Case 1 and J^+i be the set of states in the remaining graph with an edge to 

(f) If Zi ~ S, the algorithm stops, otherwise i :— i + 1 and go to the next iteration. 

Correctness and running time. The correctness of the algorithm follows from the 
correctness of the WinLose algorithm. The running time analysis of the algorithm 
is similar to ImprAlgo algorithm, and this shows the algorithm runs in 0(m • y/m) 
time. Applying the ImprWinLose algorithm bottom up on the sec decomposition of 
the MDP gives us a running time of 0{m ■ ^/Ke), where Ke is the maximum number 
of edges of an sec of the MDP. 

Theorem 6. Given an MDP with a Biichi objective, the ImprWinLose algorithm it- 
eratively computes the subsets of the almost-sure winning set and its complement, and 
in the end correctly computes the set {{!)) almost {Buchi{T)) and the algorithm runs in 
time 0{\/Ke ■ rn), where Ke is the maximum number of edges in an sec of the graph 
of the MDP. 

Symbolic implementation. The symboUc implementation of ImprWinLose algo- 
rithm is obtained in a similar fashion as SymbImprAlgo was obtained from IM- 
PRAlgo. The only additional step required is the symbolic sec computation. It follows 
from the results of (TQ\ that sec decomposition can be computed in 0{n) symbolic 
steps. In the following section we will present an improved symbolic sec computation 
algorithm. 

Corollary 1. Given an MDP with a Biichi objective, the symbolic ImprWinLose 
algorithm fSYMBlMPRWlNLoSEj iteratively computes the subsets of the almost- 
sure winning set and its complement, and in the end correctly computes the set 
ii^)) aimost{Biichi{T)) and the algorithm runs in 0{\/Ke ■ n) symbolic steps, where 
Ke is the maximum number of edges in an sec of the graph of the MDP. 

Remark 2. It is clear from the complexity of the WinLose and ImprWinLose algo- 
rithms that they would perform better for MDPs where the graph has many small scc's, 
rather than few large ones. 
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5 Improved Symbolic SCC Algorithm 

A symbolic algorithm to compute the sec decomposition of a graph in 0{n ■ logn) 
symbolic steps was presented in ||2|- The algorithm of Q was based on forward and 
backward searches. The algorithm of [TOl improved the algorithm of fl] to obtain an 
algorithm for sec decomposition that takes at most linear amount of symbolic steps. In 
this section we present an improved version of the algorithm of [10] that improves the 
constants of the number of linear symbolic steps required. We first describe the main 
ideas of the algorithm of ifTOl and then present our improved algorithm. The algorithm 
of ifTOl improves the algorithm of JD by maintaining the right order for forward sets. 
The notion of spine-sets and skeleton of a forward set was designed for this purpose. 

Spine-sets and skeleton of a forward set. Let G = (S", E) be a directed graph. Con- 
sider a finite path r = {sq, si, . . . , sg), such that for all < i < ^ — 1 we have 
(si, Si+i) S E. The path is chordless if for all < i < < ^ such that j — i > 1, there 
is no edge from Si to Sj. Let U C S. The pair ([/, s) is a spine-set of G iff G contains 
a chordless path whose set of states is U that ends in s. For a state s, let FW(s) denote 
the set of states that is reachable from s (i.e., reachable by a forward search from s). 
The set {U, t) is a skeleton of FW(s) iff f is a state in FW(s) whose distance from s is 
maximum and U is the set of states on a shortest path from s to t. The following lemma 
was shown in ifTOl establishing relation of skeleton of forward set and spine-set. 

Lemma 3 ( IIIOI ). Let G — {S, E) be a directed graph, and let FW(s) be the forward 
set of s € S. The following assertions hold: (1) If {U,t) is a skeleton of a forward-set 
FW(s), then U C FW(s). (2)If{U, t) is a skeleton q/FW(s), then {U, t) is a spine-set 
in G. 

The intuitive idea of tlie algorithm. The algorithm of ifTol is a recursive algorithm, and 
in every recursive call the sec of a state s is determined by computing FW(s), and then 
identifying the set of states in FW(s) having a path to s. The choice of the state to be 
processed next is guided by the implicit inverse order associated with a possible spine- 
set. This is achieved as follows: whenever a forward-set FW(s) is computed, a skeleton 
of such a forward set is also computed. The order induced by the skeleton is then used 
for the subsequent computations. Thus the symbolic steps performed to compute FW(s) 
is distributed over the sec computation of the states belonging to a skeleton of FW(s). 
The key to establish the linear complexity of symbolic steps is the amortized analysis. 
We now present the main procedure SCCFiND and the main sub-procedure SkelFwd 
of the algorithm from ifTOl . 

Procedures SCCFind and SkelFwd. The main procedure of the algorithm is SC- 
CFiND that calls SkelFwd as a sub-procedure. The input to SCCFind is a graph 
{S, E) and (A, B), where either [A, B) = (0, 0) or (A, B) = {U,{s }), where ([/, s) 
is a spine-set. If S is 0, then the algorithm stops. Else, (a) if [A, B) is (0, 0), then the 
procedure picks an arbitrary s from S and proceeds; (b) otherwise, the sub-procedure 
SkelFwd is invoked to compute the forward set of s together with the skeleton {U' ,s') 
of such a forward set. The SCCFiND procedure has the following local variables: 
FWSet, NewSet, NewState and SCC. The variable FWSet that maintains the forward 
set, whereas NewSet and NewState maintain U' and { s' }, respectively. The variable 
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sec is initialized to s, and then augmented with the sec containing s. The partition of 
the scc's is updated and finally the procedure is recursively called over: 

1 . the subgraph of {S, E) is induced by S\ FWSet and the spine-set of such a subgraph 
is obtained from (?7, { f }) by subtracting SCC; 

2. the subgraph of (5*, E) induced by FWSet \ SCC and the spine-set of such a sub- 
graph obtained from (NewSet, NewState) by subtracting SCC. 

The SkelFwd procedure takes as input a graph (5, E) and a state s, first it computes 
the forward set FW(s), and second it computes the skeleton of the forward set. The 
forward set is computed by symbolic breadth first search, and the skeleton is computed 
with a stack. The detailed pseudocodes are in the appendix. We will refer to this algo- 
rithm of flOl as SymbolicScc. The following result was estabhshed in fW\: for the 
proof of the constant 5, refer to the appendix of [1(J| and the last sentence explicitly 
claims that every state is charged at most 5 symbolic steps. 

Theorem 7 (QOl). Let G = {S, E) be a directed graph. The algorithm S YMBOLlcScc 
correctly computes the sec decomposition ofG in min{5- \ S\^ b-D{G) ■N(G) + N(G)} 
symbolic steps, where D{G) is the diameter of G, and N{G) is the number of scc's in 
G. 

Improved symbolic algorithm. We now present our improved symbolic sec algorithm 
and refer to the algorithm as ImprovedSymbolicScc. Our algorithm mainly modi- 
fies the sub-procedure S KELFwd. The improved version of S KELFwd procedure takes 
an additional input argument Q, and returns an additional output argument that is stored 
as a set P by the calling SCCFiND procedure. The calling function passes the set U as 
Q. The way the output P is computed is as follows: at the end of the forward search we 
have the following assignment: P :— FWSet n Q. After the forward search, the skele- 
ton of the forward set is computed with the help of a stack. The elements of the stacks 
are sets of states stored in the forward search. The spine set computation is similar to 
SkelFwd, the difference is that when elements are popped of the stack, we check if 
there is a non-empty intersection with P, if so, we break the loop and return. Moreover, 
for the backward searches in SCCFiND we initialize SCC by P rather than s. We refer 
to the new sub-procedure as ImprovedSkelFwd (detailed pseudocode in appendix). 

Correctness. Since s is the last element of the spine set U , and P is the intersection 
of a forward search from s with U , it means that all elements of P are both reachable 
from s (since P is a subset of FW(s)) and can reach s (since P is a subset of U). It 
follows that P is a subset of the sec containing s. Hence not computing the spine-set 
beyond P does not change the future function calls, i.e., the value of U' , since the 
omitted parts of NewSet are in the sec containing s. The modification of starting the 
backward search from P does not change the result, since P will anyway be included 
in the backward search. So the ImprovedSymbolicScc algorithm gives the same 
result as SymbolicScc, and the correctness follows from Theorem|7] 
SymboUc steps analysis. We present two upper bounds on the number of symbolic 
steps of the algorithm. Intuitively following are the symbolic operations that need to 
be accounted for: (1) when a state is included in a spine set for the first time in IM- 
provedSkelFwd sub-procedure which has two parts: the first part is the forward 
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search and the second part is computing the skeleton of the forward set; (2) when a 
state is already in a spine set and is found in forward search of ImprovedSkelFwd 
and (3) the backward search for determining the sec. We now present the number of 
symbohc steps analysis for ImprovedSymbolicScc. 

1. There are two parts of ImprovedSkelFwd, (i) a forward search and (ii) a back- 
ward search for skeleton computation of the forward set. For the backward search, 
we show that the number of steps performed equals the size of NewSet computed. 
One key idea of the analysis is the proof where we show that a state becomes part 
of spine-set at most once, as compared to the algorithm of ifTOll where a state can 
be part of spine-set at most twice. Because, when it is already part of a spine-set, it 
will be included in P and we stop the computation of spine-set when an element of 
P gets included. We now split the analysis in two cases: (a) states that are included 
in spine-set, and (b) states that are not included in spine-set. 

(a) We charge one symbolic step for the backward search of ImprovedSkelFwd 
(spine-set computation) to each element when it first gets inserted in a spine- 
set. For the forward search, we see that the number of steps performed is the 
size of spine-set that would have been computed if we did not stop the skeleton 
computation. But by stopping it, we are only omitting states that are part of 
the sec. Hence we charge one symbolic step to each state getting inserted into 
spine-set for the first time and each state of the sec. Thus, a state getting inserted 
in a spine-set is charged two symbolic steps (for forward and backward search) 
of ImprovedSkelFwd the first time it is inserted. 

(b) A state not inserted in any spine-set is charged one symbolic step for backward 
search which determines the sec. 

Along with the above symbolic steps, one step is charged to each state for the 
forward search in ImprovedSkelFwd at the time its sec is being detected. 
Hence each state gets charged at most three symbolic steps. Besides, for computing 
NewState, one symbolic step is required per sec found. Thus the total number of 
symbolic steps is bounded by 3 • |5| + N{G), where N{G) is the number of see's 
of G. 

2. Let D* be the sum of diameters of the see's in a G. Consider a sec with diameter 
d. In any sec the spine-set is a shortest path, and hence the size of the spine-set 
is bounded by d. Thus the three symbolic steps charged to states in spine-set con- 
tribute to at most 3 • d symbolic steps for the sec. Moreover, the number of iterations 
of forward search of ImprovedSkelFwd charged to states belonging to the sec 
being computed are at most d. And the number of iterations of the backward search 
to compute the sec is also at most d. Hence, the two symbolic steps charged to 
states not in any spine-set also contribute at most 2 • d symbolic steps for the sec. 
Finally, computation of NewSet takes one symbolic step per sec. Hence we have 
5 ■ d + 1 symbolic steps for a sec with diameter d. We thus obtain an upper bound 
of 5D* + N{G) symboHc steps. 

It is straightforward to argue that the number of symbolic steps of ImprovedSCCFind 
is at most the number of symbolic steps of SCCFlND. The detailed pseudocode and 
running time analysis is presented in the appendix. 
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Theorem 8. Let G — (5, E) be a directed graph. The algorithm ImprovedSymbol- 
ICSCC correctly computes the sec decomposition of G in niin{ 3 • 15*1 + ■N{G),5 ■ 
D*{G) + N{G) } symbolic steps, where D*{G) is the sum of diameters of the scc's of 
G, and N{G) is the number of scc's in G. 

Remark 3. Observe that in the worst case SCCFiND takes 5 • n symboHc steps, whereas 
ImprovedSCCFind takes at most 4 • n symboHc steps. Thus our algorithm improves 
the constant of the number of linear symbolic steps required for symbolic sec decom- 
position. 

6 Experimental Results 

In this section we present our experimental results. We first present the results for sym- 
bolic algorithms for MDPs with Biichi objectives and then for symbolic sec decompo- 
sition. 

Symbolic algorithm for MDPs with Biichi objectives. We implemented all the sym- 
bolic algorithms (including the classical one) and ran the algorithms on randomly gener- 
ated graphs. If we consider arbitrarily randomly generated graphs, then in most cases it 
gives rise to trivial MDPs. Hence we generated large number of MDP graphs randomly, 
first chose the ones where all the algorithms required the most number of symbolic 
steps, and then considered random graphs obtained by small uniform perturbations of 
them. Our results of average symbolic steps required are shown in Table [T] and show 
that the new algorithms perform significantly better than the classical algorithm. The 
running time comparison is given in Table |2] 



Number of states 


Classical 


SymbImprAlgo 


SmDvSymbImprAlgo 


SymbImprWinLose 


5000 


16508 


3382 


3557 


4007 


10000 


57438 


6807 


7489 


7146 


20000 


121376 


11110 


11882 


12519 



Table 1. The average symbolic steps required by symbolic algorithms for MDPs with 
Biichi objectives. 



Number of states 


Classical 


SymbImprAlgo 


SmDvSymbImprAlgo 


SymbImprWinLose 


5000 


29.8 


8.5 


8.9 


10.7 


10000 


316.1 


53.5 


55.4 


60.9 


20000 


1818.4 


224.1 


228.4 


268.7 



Table 2. The average running time required in sec by symbolic algorithms for MDPs 
with Biichi objectives. 



Symbolic sec computation. We implemented the symbolic sec decomposition algo- 
rithm from 1, 10] and our new symbolic algorithm. We ran the algorithms on randomly 
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generated graphs. Again arbitrarily randomly generated graphs in many cases gives rise 
to graphs that are mostly disconnected or completely connected. Hence we generated 
random graphs by first constructing a topologically sorted order of the scc's and then 
adding edges randomly respecting the topologically sorted order Our results of average 
symbolic steps are shown in Table |3] and shows that our new algorithm performs better 
(around 15% improvement). The running time comparison is shown in Tab|4] 



Number of states 


Algorithm from 1 10 1 


Our Algorithm 


Percentage Improvement 


10000 


1045 


877 


16.06 


25000 


2642 


2262 


14.38 


50000 


6298 


5398 


14.27 



Table 3. The average symbolic steps required for sec computation. 



Number of states 


Algorithm from 1 10] 


Our Algorithm 


Percentage Improvement 


10000 


16.6 


13.6 


18.08 


25000 


96.6 


80.1 


17.08 


50000 


364.1 


308.9 


15.15 



Table 4. The average running time required in sec for sec computation. 
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Appendix 



7 Details of Section |3] 

Pseudocode of ImprAlgo. We present the pseudocode of the algorithm ImprAlgo in 
Algorithm[T] The symbolic implementation of the algorithm is obtained as described in 
the description of SymbImprAlgo. The optimized version SmDvSymbImprAlgo 
is also obtained following the description in Subsection 13. 3 1 

Details of correctness and symbolic step analysis of SmDvSymbImprAlgo (i.e. of 
subsection 13.3) . We now present the details of correctness and symbolic steps analysis 
for SmDvSymbImprAlgo. 

Correctness and symbolic steps analysis. We present the correctness and number of 
symbolic steps required analysis for the algorithm SmDvSymbImprAlgo. The cor- 
rectness analysis is same as ImprAlgo and the only change is as follows (we describe 
iteration i): (a) if in Case 2 we obtain a set Pj = Pj+i and its intersection with Ui is 
empty, then there is no path from Pj to Ui and since Ti C Ui, it follows that there is 
no path from Pj to Uf, (b) if Pj O Ui ^ 0, then since Ui is obtained as the backward 
exploration from T^, every state in Ui has a path to Ti, and it follows that there is a 
path from the starting state of Pj to Ui and hence to Ti, and (c) if J7i = Pre(C/i), then 
Ui is the set of states that can reach Ti and all the other states can be removed. Thus 
the correctness follows similar to the arguments for ImprAlgo. The key idea of the 
running time analysis is as follows: 

1. Case 1 of the algorithm is same to Case 1 of SymbImprAlgo, and in Case 2 
the algorithm also runs like SymbImprAlgo, but for every symbolic step (Post 
computation) of SymbImprAlgo, there is an additional (Pre) computation. Hence 
the total number of symboUc steps of SmDvSymbImprAlgo is at most twice the 
number of symbolic steps of SymbImprAlgo. However, the optimized step of 
maintaining the set Ui which includes Ti may allow to stop several of the forward 
exploration as they may intersect with Ui earlier than intersection with Ti. 

2. Case 1 of the algorithm is same as in Case 1 of the classical algorithm. In Case 2 of 
the algorithm the backward exploration step is the same as the classical algorithm, 
and (i) for every Pre computation, there is an additional Post computation and 
(ii) for every check whether Ui — Pre(?7i), there is a check whether Pj = Pj+i or 
Pj C\Ui^%. It follows that the total number of symbolic steps of Case 1 and Case 
2 over all iterations is at most twice the number of symbolic steps of the classical 
algorithm. The cardinality computation takes additional 0{m) symbolic steps over 
all iterations. 

Hence it follows that SmDvSymbImprAlgo takes at most 

min{ 2 • SymbStep(SYMBlMPRALGO), 2 • SymbStep(CLASSlCAL) + 0(m) } 
symbolic steps. 
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Algorithm 1 ImprAlgo 



Input: An MDP G = {{S,E), [Si, Sp),S) with Buchi set T. 

Output: ((1)) almost (Buchi(T)), i.e., tlie almost-sure winning set for player 1. 

1. i ■- 0; So ■- S; Eo := E; To := T; 

2. Lo ■- Zo ■- Jo ■- 0; 

3. if (I Ji I > ^/rn oi i — 0) then 

3.1. Yi := Reach(ri, {Si, Ei)); (i.e., compute the set Yi that can reach Ti in the graph {Si, Ei)) 

3.2. Q, ■- S, \ Y,; 

3.3. if {Qr = 0) then goto line 6; 

3.4. else goto hne 5; 

4. else (i.e., Ji < y/m) 

4.1. for each s € Ji 

4.1.1. DFSi,s ■- s; (initializing DFS-trees) 

4.2. for each s € Ji 

4.2.1. Do 1 step of DPS from DFSi^s, unless it has encountered a state from Ti 

4.2.2. If DPS encounters a state from T, mark that DPS as stopped 

4.2.3. if DPS completes without meeting T then 

4.2.3.1. Q^ ■- DFS^,s■, 

4.2.3.2. goto line 5; 

4.2.4. if all DPSs meet Ti then 
4.2.4.1. goto line 6; 

5. Removal of attractor of Qi in the following steps 

5.1 Z^+i ■- U Attrn{Q^, {S^, Ei), {Si n S^, Sp n S,)); 
5.2. Si+i := Si \ Zi+i; Ei+i := Ei (1 Si+i x Si+i; 

5.4. if the last goto call from step 3.4 (i.e. Case 1 is executed) then 
5.4.1 := AttrR{Qi, (S„ Ei), {Si n S^, Sp n S,)); 

5.5. else Li+i ■- Li U AttrR{Qi, {Si, Ei), {Si n Si, Sp n Si)); 
5.6 J.+i := E-\L,+i) n S^+i; 

5.7. j := i + 1; 

5.8. goto line 3; 

6. return S\Zi; 



8 Details of Section H 

Pseudocode of WinLose and ImprWinLose. We present the pseudocode of the al- 
gorithm WinLose in Algorithm |2] The pseudocode of the algorithm ImprWinLose 
is presented in Algorithmic] 

Details related to the correctness of ImprWinLose. The correctness proof of Im- 
prWinLose is similar as the correctness argument of WinLose algorithm. One addi- 
tional care requires to be taken for Case 2: we need to show that when we terminate the 
lockstep DFS search in Case 2, then we obtain a bottom sec. First, we observe that in 
iteration i, when Case 2 is executed, each bottom sec must contain a state from Ji, since 
it was not a bottom sec in the last execution of Case 1 . Second, among all the lockstep 
DPSs, the first one that terminates must be a bootm sec because the DFS search from 
a state of Ji that does not belong to a bottom sec explores states of bottom scc's below 
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Algorithm 2 WinLose 



Input: An MDP G = {{S,E), [Si, Sp),S) with Buchi set T. 

Output: ((l))ai,„os((Buchi(T)), i.e., tlie almost-sure winning set for player 1. 

1. W ■- Wi ■- W2 ■- 

2. while(M/ / 5") do 

2.1. sees :=SCC-Decomposition(S' \ W) (i.e. sec decomposition of the graph induced by 5 \ W) 

2.2. for each C in SCCS 

2.2.1. if {E{C) C CUW) then (checks if C is a bottom sec in graph induced by S\W) 
2.2.1.1.ifCnr/0or_B(C)nM /0then 

2.2.1.1.1. Wi —WiUC 
2.2.1.2. else 

2.2.1.2.1.1^2 ■-W2UC 
23. Wi := Attri{Wi,{S,E),{Si,Sp)) 
2A.W2 := AttrR{W2,{S,E),{Si,Sp)) 
2.5. W — WiU W2 

3. return Wi 



it. Since Case 2 stops wlien tiie first DFS terminates we obtain a bottom sec. Tiie rest 
of the correctness proof is as for the WinLose algorithm. The running time analysis is 
similar to the running time analysis of ImprAlgo. 

Details related to correctness of SymbImprWinLose. The correctness proof of 
SymbImprWinLose is similar to ImprWinLose algorithm. For the coiTectness of 
the SymbImprWinLose algorithm we again need to take care that when we terminate 
in Case 2, then we have identified a bottom sec. Note that for symbolic step forward 
search we cannot guarantee that the forward search that stops first gives a bottom sec. 
For Case 2 of the SymbImprWinLose we do in lockstep both symbolic forward and 
backward searches, stop when both the searches stop and gives the same result. Thus we 
ensure when we terminate an iteration of Case 2 we obtain a bottom sec. The correct- 
ness then follows from the correctness arguments of WinLose and ImprWinLose. 
The symbolic steps required analysis is same as for SymbImprAlgo. The desired 
result follows. 



9 Details of Section |5] 

The pseudocode of SCCFiND is formally given as Algorithm|4] The coiTectness anal- 
ysis and the analysis of the number of symbolic steps is given in ifTOl . The pseudocode 
of ImprovedSCCFind is formally given as Algorithm |5] The main changes of IM- 
PROVEDSCCFind from SCCFind are as follows: (1) instead of SkelFwd the algo- 
rithm ImprovedSCCFind calls procedure ImprovedSkelFwd that returns an ad- 
ditional set P and ImprovedSkelFwd is invoked with an additional argument that is 
U; (2) in line 4 of ImprovedSCCFind the set SCC is initiahzed to P instead of s. The 
main difference of ImprovedSkelFwd from SkelFwd is as follows: (1) the set P is 
computed in Une 4 of ImprovedSkelFwd as FWSet n Q, where Q is the set passed 
by ImprovedSCCFind as the argument; and (2) in the while loop it is checked if the 
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element popped intersects with P and if yes, then the procedure breaks the while loop. 
The correctness argument from the correctness of SCCFiND is already shown in the 
main paper. 

Symbolic steps analysis. We now present the detailed symbolic steps analysis of the 
algorithm. As noted in Section l372l common symbolic operations on a set of states are 
Pre, Post and CPre. We note that these operations involve symbolic sets of 2 • log(n) 
variables, as compared to symbolic sets of log(ri) variables required for operations such 
as union, intersection and set difference. Thus only Pre, Post and CPre are counted as 
symbolic steps, as done in |10|. The total number of other symbolic operations is also 
0{\S\). We note that only Hnes 5 and 10 of ImprovedSCCFind and lines 3.3 and 7.3 
of ImprovedSkelFwd involve Pre and Post operations. 

In the following, we charge the costs of these lines to states in order to achieve the 
3 • jS'l + N{G) bound for symbolic steps. We define subspine-set as NewSet returned 
by ImprovedSkelFwd and show the following result. 

Lemma 4. For any spine-set U and its end vertex u, T is a subspine-set iffU\T C 
SCC(u). 

Proof. Note that while constructing a subspine-set T, we stop the construction when 
we find any state w in a subspine-set. Now clearly since v E U, there is a path from 

V to u. Also, since we found this state in FW{u), there is a path from u to v. Hence, 

V G SCC(u). Also, each state that we are omitting by stopping construction of T has 
the property that there is a path from u to that state and a path from that state to v. This 
implies that all the states we are omitting in construction of T are in SCC(u). I 

Note that since we pass NewSet \ SCC in the subsequent call to ImprovedSC- 
CFind, it will actually be a spine set. In the following lemma we show that any state 
can be part of subspine-set at most once, as compared to twice in the SCCFiND pro- 
cedure in [10|. This lemma is one of the key points that lead to the improved symbolic 
steps required analysis. 

Lemma 5. Any state v can be part of subspine-set at most once. 

Proof. In HP], the authors show that any state can be included in spine sets at most 
twice in SkelFwd. The second time it is included is in line 6 of SkelFwd when the 
SCC of that state is to be found. In contrast, ImprovedSkelFwd checks intersection 
of the subspine-set being constructed with the set P that contains the states of this SCC 
which are already in a subspine-set. When this happens, it stops the construction of 
spine set. Now if v is already included in the spine set, then it will be part of P and 
would not be included in subspine-set again. Hence, v can be part of subspine-set at 
most once. I 

Lemma 6. States added in SCC by iteration of line 5 o/ImprovedSCCFind are ex- 
actly the states which are not part of any subspine-set. 

Proof We see that in line 5 of ImprovedSCCFind, we start from SCC = P and then 
we find the SCC by backward search. Also, P has all the states from SCC which are 
part of subspine-set. Hence, the extra states that are added in SCC are states which are 
never included in a subspine-set. I 
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Charging symbolic steps to states. We now consider three cases to chai-ge symbolic 
steps to states and scc's. 

1 . Charging states included in subspine-set. First, we see that the number of times 
the loop of Une 3 in ImprovedSkelFwd is executed is equal to the size of the 
spine set that SkelFwd would have computed. Using Lemma ID we can charge 
one symbolic step to each state of the subspine-set and each state of the SCC that 
is being computed. Now, the number of times line 7.3 of ImprovedSkelFwd is 
executed equals the size of subspine-set that is computed. Hence, we charge one 
symbolic step to each state of subspine-set for this line. 

Now we summarize the symbolic steps charged to each state which is part of some 
subspine-set. First time when a state gets into a subspine-set, it is charged two steps, 
one for Hne 3.3 and one for line 7.3 of ImprovedSkelFwd. If its SCC is not found 
in the same call to ImprovedSCCFind, then it comes into action once again when 
its SCC is being found. By Lemma|5] it is never again included in a subspine set. 
Hence in this call to ImprovedSkelFwd, it is only charged one symbolic step for 
line 3.3 and none for line 7.3 as line 7.3 is charged to states that become part of the 
newly constructed subspine-set. Also because of Lemma |6] since this state is in a 
subspine-set, it is not charged anything for line 5 of ImprovedSCCFind. Hence, 
a state that occurs in any subspine-set is charged at most three symbolic steps. 

2. Charging states not included in subspine-set. For line 5 of IMPROVEDSCCFIND, 
the number of times it is executed is the number of states that are added to SCC 
after initialization to SCC = P. Using Lemma |6] we charge one symbolic step 
to each state of this SCC that is never a part of any subspine-set. Also, we might 
have charged one symbolic step to such a state for line 3.3 of ImprovedSkelFwd 
when we called it. Hence, each such state is charged at most two symbolic steps. 

3. Charging SCCi. For line 10 of ImprovedSCCFind, we see that it is executed 
only once in a call to ImprovedSCCFind that computes a SCC. Hence, the total 
number of times line 10 is executed equals N{G), the number of SCCs of the graph. 
Hence, we charge each SCC one symbolic step for this line. 

The above argument shows that the number of symbolic steps that the algorithm Im- 
provedSCCFind requires is at most 3 • [S"! + N{G). 
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Algorithm 3 ImprWinLose 



Input: An MDP G={{S,E), (Si ,Sp),S) with Buchi set T. 

Output: ((1)) ahnost (Buchi(T)), i.e., the almost-sure winning set for player 1. 

1. i := 0; So := S; Eo := E; To := T; 

2. Wi := W2 ■■= Lo := Zo := Jo := 0; 

3. if {\Ji\ > or i = 0) then 

3.1. sees :=SCC-Decomposition(Si) (sec decomposition of graph induced by Si) 

3.2. for each C in SCCS 

3.2.1. if {Ei{C) C C) then (checks if C is a bottom sec in graph induced by Si) 

3.2.1.1. if C n T or £(C) n m 7^ then 
3.2.1.1.1. Wi ■.= WiLlC 

3.2.1.2. else 
3.2.1.2.1.1^2 ■.= W2UC 

3.3. goto line 5 

4. else (i.e., Ji < y/m) 

4.1. for each s G Ji 

4.1.1. DFSi,s ■■= s (initiahzing DFS-trees) 

4.2. for each s G Ji 

4.2.1. Do 1 step of DPS from DFSi,s 

4.2.2. if DPS completes then 

4.2.2.1. C := DPS,,, 

4.2.2.2. if C n T ^ or E{C) nWiy^0 then 
4.2.2.2.1. Wi := W^i U C7 

4.2.2.3. else 

4.2.2.3.1.1^2 ■.= W2UC 

4.2.2.4. goto line 5 

5. Removal of Wi and W2 states in the following steps 

5.1. Wi := AttniWi, {Si, Ei), {Si n Si, Sp n Si)) 

5.2. W2 := AttrR{W2, (S„ Ei), {Si n Si, Sp n Si)) 

5.3. Z,+i ~ Z^UWiU W2 

5.4. Si+i := Si \ Zi+i; Et+i := Ei n Si+i x Si+i 

5.5. if the last goto call was from line 3.3 then 
5.5.1. Li+i := Zi+i \ Zi 

5.6. else 

5.6.1. L,+ i := LiU{Zi+i\Zi) 

5.7. Ji+i :— E ^(Li+i) n Si+i 

5.8. if Zi+i = Sthen 
5.8.1. goto Une 6 

5.9. i ~ goto line 3 

6. return Wi 
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Algorithm 4 SCCFind 

Input: {S, E, {U, s)), i.e., a graph (S, E) with spine set (U, s). 
Output: SCCPartition i.e. the set of SCCs of the graph {S, E) 

I. if (S' = 0)then 
1.1 return; 

2. if {U = 0) tlien 
2.1 s := pick(S) 

3. (FWSet, NewSet, NewState) — SKELFWD(S',i;, s) 

4. sec = s 

5. wWle (((Pre(SCC) n FWSet) \ SCC) ^ 0) do 
5.1 SCC := SCC U (Pre(SCC) n FWSet) 

6. SCCPartition — SCCPartition U {SCC} 

(Recursive call on S \ FWSet) 

7. S" := S \ FWSet 

8. E' :=En {S' X S') 

9. U' :=U\ SCC 

10. s' := Pre(SCC nU)n{S\ SCC) 

II. SCCPartition := SCCPartitionU SCCFind(S', S', (U',s')) 
(Recursive call on FWSet \ SCC) 

12. S' := FWSet \ SCC 

13. E' ■.= En{S' X S") 

14. U' := NewSet \ SCC 

15. s' := NewState \ SCC 

16. SCCPartition := SCCPartitionU SCCFind(S", {U',s')) 

17. Return SCCPartition 

Procedure SkelFwd 

Input: (S, E, s), i.e., a graph (5, E) with a state s £ S. 
Output: (FWSet, NewSet, NewState), 

i.e. forward set FWSet, new spine-set NewSet and NewState € NewSet 

1. Let stack be an empty stack of sets of nodes 

2. L--S 

3. while (L ^ 0) do 

3.1 Push(stack,L) 

3.2 FWSet := FWSet U L 

3.3 L ■- Post(L) \ FWSet 

4. L := Pop(stack) 

5. NewSet := NewState := pick(L) 

6. wMle (stack ^ 0) do 

6.1 L := Pop(stack) 

6.2 NewSet := NewSet U pick(Pre(NewSet) n L) 
7. return (FWSet, NewSet, NewState) 
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Algorithm 5 ImprovedSCCFind 

Input: {S, E, (U, s)), i.e., a graph {S, E) with spine set {U, s). 
Output: SCCPartition, the set of SCCs of the graph {S, E) 

1. if (5* = 0) then 
1.1 return; 

2. if {U = 0) then 
2.1 s := pick(S') 

3. (FWSet, NewSet, NewState, P) := ImprovedSkelFwd(S',S, U, s) 

4. sec = P 

5. while (((Pre(SCC) n FWSet) \ SCC) / 0) do 
5.1 SCC := SCC U (Pre(SCC) n FWSet) 

6. SCCPartition := SCCPartition U {SCC} 

(Recursive call on 5 \ FWSet) 

7. S' ■-S\ FWSet 

8. E' :=En{S' X S') 

9. U' ■.= U\ SCC 

10. s' := Pre(SCC nU)n{S\ SCC) 

11. SCCPartition := SCCPartitionU ImprovedSCCFind(S", E', {U' , s')) 
(Recursive call on FWSet \ SCC) 

12. S' := FWSet \ SCC 

13. E' ■.= En {S' X S') 

14. U' := NewSet \ SCC 

15. s' := NewState \ SCC 

16. SCCPartition := SCCPartitionU ImprovedSCCFind(S', E', {U', s')) 



Procedure ImprovedSkelFwd 

Input: (S, E, Q, s), i.e., a graph {S, E) with a set Q and a state s € S. 
Output: {FWSet, NewSet, NewState, P) 

1. Let stack be an empty stack of sets of nodes 

2. L--S 

3. while (L / 0) do 

3.1 Push(stack,L) 

3.2 FWSet := FWSet U L 

3.3 L := Post(L) \ FWSet 

4. P ■- FWSet n Q 

5. L := Pop(stack) 

6. NewSet := NewState := pick(L) 

7. while (stack ^ 0) do 

7.1 L := Pop(stack) 

7.2 if (L n P / 0) then 
7.2.1 break while loop 

7.3 else NewSet := NewSet U pick(Pre(NewSet) n L) 
8. return (FWSet, NewSet, NewState, P) 
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